Repeated measures of implementation variables.

It is commonly acknowledged that implementation work is long-term and contextual in nature and often takes years to accomplish. Repeated measures are needed to study the trajectory of implementation variables over time. To be useful in typical practice settings, measures that are relevant, sensitive, consequential, and practical are needed to inform planning and action. If implementation independent variables and implementation dependent variables are to contribute to a science of implementation, then measures that meet these criteria must be established. This exploratory review was undertaken to "see what is being done" to evaluate implementation variables and processes repeatedly in situations where achieving outcomes was the goal (i.e., more likely to be consequential). No judgement was made about the adequacy of the measure (e.g., psychometric properties) in the review. The search process resulted in 32 articles that met the criteria for a repeated measure of an implementation variable. 23 different implementation variables were the subject of repeated measures. The broad spectrum of implementation variables identified in the review included innovation fidelity, sustainability, organization change, and scaling along with training, implementation teams, and implementation fidelity. Given the long-term complexities involved in providing implementation supports to achieve the full and effective use of innovations, repeated measurements of relevant variables are needed to promote a more complete understanding of implementation processes and outcomes. Longitudinal studies employing repeated measures that are relevant, sensitive, consequential, and practical should become common if the complexities involved in implementation are to be understood.

The Teaching-Family Model: The First 50 Years.

The Teaching-Family Model was perhaps the first "evidence-based program" in human services. This article describes the development of the treatment model, the failure of the first attempts to replicate the treatment model, the discovery of larger units for replication, the modest success of first attempts to replicate larger units, and the eventual success of replications. The Teaching-Family Model is a testament to the sustainability (and continual improvement) of innovation and implementation methods and the value of the Teaching-Family Association for sustaining a community of practice and for managing the practitioner fidelity and organization fidelity data systems nationally. The benefits of applied behavior analysis and the implications for a new science of implementation for having research purposefully used in practice are explored.

Selecting an EBP to reduce long-term foster care: lessons from a university-child welfare agency partnership.

A growing implementation literature outlines broad evidence-based practice implementation principles and pitfalls. Less robust is knowledge about the real-world process by which a state or agency chooses an evidence-based practice to implement and evaluate. Using a major U.S. initiative to reduce long-term foster care as the case, this article describes three major aspects of the evidence-based practice selection process: defining a target population, selecting an evidence-based practice model and purveyor, and tailoring the model to the practice context. Use of implementation science guidelines and lessons learned from a unique private-public-university partnership are discussed.

Usability testing, initial implementation, and formative evaluation of an evidence-based intervention: lessons from a demonstration project to reduce long-term foster care.

The field of child welfare faces an undersupply of evidence-based interventions to address long-term foster care. The Permanency Innovations Initiative is a five-year federal demonstration project intended to generate evidence to reduce long stays in foster care for those youth who encounter the most substantial barriers to permanency. This article describes a systematic and staged approach to implementation and evaluation of a PII project that included usability testing as one of its key activities. Usability testing is an industry-derived practice which analyzes early implementation processes and evaluation procedures before they are finalized. This article describes the iterative selection, testing, and analysis of nine usability metrics that were designed to assess three important constructs of the project's initial implementation and evaluation: intervening early, obtaining consent, and engaging parents. Results showed that seven of nine metrics met a predetermined target. This study demonstrates how findings from usability testing influenced the initial implementation and formative evaluation of an evidence-supported intervention. Implications are discussed for usability testing as a quality improvement cycle that may contribute to better operationalized interventions and more reliable, valid, and replicable evidence.

When evidence is not enough: the challenge of implementing fall prevention strategies.

PROBLEM: As the evidence-based movement has advanced in public health, changes in public health practices have lagged far behind creating a science to service gap. For example, science has produced effective falls prevention interventions for older adults. It now is clearer WHAT needs to be done to reduce injury and death related to falls. However, issues have arisen regarding HOW to assure the full and effective uses of evidence-based programs in practice. SUMMARY: Lessons learned from the science and practice of implementation provide guidance for how to change practices by developing new competencies, how to change organizations to support evidence-based practices, and how to change public health systems to align system functions with desired practices. The combination of practice, organization, and system change likely will produce the public health benefits that are the promise of evidence-based falls prevention interventions. IMPACT ON PUBLIC HEALTH: For the past several decades, the emphasis has been solely on evidence-based interventions. Public health will benefit from giving equal emphasis to evidence-based implementation. IMPACT ON INDUSTRY: We now have over two decades of research on the effectiveness of fall prevention interventions. The quality of this research is judged by a number of credible international organizations, including the Cochrane Collaboration (http://www.cochrane.org/), the American and British Geriatrics Societies, and the Campbell Collaboration (http://www.campbellcollaboration.org/). These international bodies were formed to ponder and answer questions related to the quality and relevance of research. These developments are a good first step. However, while knowing WHAT to do (an evidence-based intervention) is critical, we also need to know HOW to effectively implement the evidence. Implementation, organization change, and system change methods produce the conditions that allow and support the full and effective use of evidence-based interventions. It is time to focus on utilization of implementation knowledge in public health. Without this focus the vast amount on new evidence being generated on the prevention of falls and related injuries among older adults will have little impact on their health and safety.

Mobilizing communities for implementing evidence-based youth violence prevention programming: a commentary.

Evidence-based programs have struggled for acceptance in human service settings. Information gleaned from these experiences indicates that implementation is the missing link in the science to service chain. The science and practice of implementation is progressing and can inform approaches to full and effective uses of youth violence prevention programs nationally. Implementation Teams that know (a) innovations, (b) implementation best practices, and (c) improvement cycles are essential to mobilizing support for successful uses of evidence-based programs on a socially significant scale. The next wave of development in implementation science and practice is underway: establishing infrastructures for implementation to make implementation expertise available to communities nationally. Good science, implemented well in practice, can benefit all human services, including youth violence prevention.

Research utilization among children's mental health providers.

BACKGROUND: Children with emotional and behavioural disorders should be able to count on receiving care that meets their needs and is based on the best scientific evidence available, however, many do not receive these services. Implementation of evidence-based practice (EBP) relies, in part, on the research utilization practices of mental health care providers. This study reports on a survey of research utilization practices among 80 children's mental health (CMH) service provider organizations in Ontario, Canada. METHODS: A web-based survey was distributed to 80 CMH service provider organizations, to which 51 executive directors and 483 children's mental health practitioners responded. Research utilization was assessed using questions with Likert-type responses based on the Canadian Health Services Research Foundation's Four-A's approach: access, assess, adapt, apply. RESULTS: There was general agreement among executive directors and practitioners regarding the capacity of their organizations to use - access, assess, adapt, and apply - research evidence. Overall, both groups rated their organizations as using research information 'somewhat well.' The low response rate to the practitioner survey should be noted. CONCLUSION: These findings provide a useful benchmark from which changes in reported research utilization in the Ontario CMH sector can be tracked over time, as a function of EBP training and implementation initiatives, for instance. The need to improve access to research evidence should be addressed because it relates to the eventual implementation and uptake of evidence-based practices. Communities of practice are recommended as a strategy that would enable practitioners to build capacity in their adaptation and application of research evidence.